Papers with Evaluation frameworks
Towards Multi-dimensional Evaluation of LLM Summarization across Domains and Languages (2025.acl-long)
Copied to clipboard
Hyangsuk Min, Yuho Lee, Minjeong Ban, Jiaqi Deng, Nicole Hee-Yeon Kim, Taewon Yun, Hang Su, Jason Cai, Hwanjun Song
| Challenge: | Existing evaluation frameworks for text summarization lack domain-specific assessment criteria and are predominantly English-centric. |
| Approach: | They propose a multi-dimensional, multi-domain evaluation of summarization in English and Chinese that incorporates specialized assessment criteria for each domain and leverages a debate system to enhance annotation quality. |
| Outcome: | The proposed evaluation framework provides a multi-dimensional, multi-domain evaluation of summarization in English and Chinese. |
A Survey on Evaluation of LLM-based Agents (2026.findings-acl)
Copied to clipboard
Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, Michal Shmueli-Scheuer
| Challenge: | This paper provides the first comprehensive survey of evaluation methods for LLM-based agents . LLMs are static, having fixed knowledge, and confined to text-to-text interaction. |
| Approach: | They analyze the evaluation of LLM-based agents across five perspectives . they identify current trends and key gaps in evaluation methods . |
| Outcome: | The proposed evaluation frameworks and tools are based on five perspectives . the results highlight current trends and identify gaps in future research . |